Ieee Transactions on Consumer Electronics

In next generation of smart computing environment, myriads of networked resources are pervasive and embedded into everyday consumer products and appliances. Still, most controller systems provide separate control user interfaces for different uses when users inevitably face an increased number of interfaces for control and interaction tasks in such environment. To simplify burdens of user and reduce complexities in control tasks, we propose a universal appliance controller based on mobile device with two selective interaction modes. First mode is a network scanning-based indirect interaction mode which provides fast and reliable connection to the appliances with specific filters in the networked environment. Second mode is a camera-based direct interaction mode which provides an intuitive interface as an input device. These two modes are interchangeable for different tasks and according to user’s preference. Moreover, control user interface is personalized with preference described in user profile to reflect the user of the system better. To verify usefulness of our approach, we present a prototype in a smart home test bed. Through an initial user study, we demonstrate usability of our prototype through a set of mobile interaction tasks. Moreover, we note usability issues, limitations of our work and analyze the participants’ feedback. Index Terms — universal controller, service discovery, mobile interaction, camera-based interaction.